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Introduction 


The Information Commissioner (the Commissioner) is calling for evidence 
and views on the Age Appropriate Design Code (the Code). 


The Code is a requirement of the Data Protection Act 2018 (the Act). The 
Act supports and supplements the implementation of the EU General Data 
Protection Regulation (the GDPR). 


The Code will provide guidance on the design standards that the 
Commissioner will expect providers of online ‘Information Society 
Services’ (ISS), which process personal data and are likely to be accessed 
by children, to meet. Once it has been published, the Commissioner will 
be required to take account of any provisions of the Code she considers to 
be relevant when exercising her regulatory functions. The courts and 
tribunals will also be required to take account of any provisions they 
consider to be relevant in proceedings brought before them. The Code 
may be submitted as evidence in court proceedings. 


Further guidance on how the GDPR applies to children’s personal data can 
be found in our guidance Children and the GDPR. It will be useful to read 
this before responding to the call for evidence, to understand what is 
already required by the GDPR and what the ICO currently recommends as 
best practice. In drafting the Code the ICO may consider suggestions that 
reinforce the specific requirements of the GDPR, or its overarching 
requirement that children merit special protection, but will disregard any 
suggestions that fall below this standard. 


The Commissioner will be responsible for drafting the Code. The Act 
provides that the Commissioner must consult with relevant stakeholders 
when preparing the Code, and submit it to the Secretary of State for 
Parliamentary approval within 18 months of 25 May 2018. She will publish 
the Code once it has been approved by Parliament. 


This call for evidence is the first stage of the consultation process. The 
Commissioner seeks evidence and views on the development stages of 
childhood and age-appropriate design standards for ISS. The 
Commissioner is particularly interested in evidence based submissions 
provided by: bodies representing the views of children or parents; child 
development experts; providers of online services likely to be accessed by 
children, and trade associations representing such providers. She 
appreciates that different stakeholders will have different and particular 
areas of expertise. The Commissioner welcomes responses that are 
limited to specific areas of interest or expertise and only address 
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questions within these areas, as well as those that address every question 
asked. She is not seeking submissions from individual children or parents 
in this call for evidence as she intends to engage with these stakeholder 
groups via other dedicated and specifically tailored means. 


The Commissioner will use the evidence gathered to inform further work 
in developing the content of the Code. 


The scope of the Code 


The Act affords the Commissioner discretion to set such standards of age 
appropriate design as she considers to be desirable, having 

regard to the best interests of children, and to provide such guidance as 
she considers appropriate. 


In exercising this discretion the Act requires the Commissioner to have 
regard to the fact that children have different needs at different ages, and 
to the United Kingdom's obligations under the United Nations Convention 
on the Rights of the Child. 


During Parliamentary debate the Government committed to supporting 
the Commissioner in her development of the Code by providing her with a 
list of ‘minimum standards to be taken into account when designing it.’ 
The Commissioner will have regard to this list both in this call for 
evidence, and when exercising her discretion to develop such standards 
as she considers to be desirable 


In developing the Code the Commissioner will also take into account that 
the scope and purpose of the Act, and her role in this respect, is limited to 
making provision for the processing of personal data. 


Responses to this call for evidence must be submitted by 19 September 
2018. You can submit your response in one of the following ways: 


Online 


Download this document and email to: 


childrenandtheGDPR@ICO.org.uk 


Print off this document and post to: 

Age Appropriate Design Code call for evidence 
Engagement Department 

Information Commissioner's Office 

Wycliffe House 

Water Lane 
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Wilmslow 
Cheshire SK9 5AF 


If you would like further information on the call for evidence please 
telephone 0303 123 1113 and ask to speak to the Engagement 
Department about the Age Appropriate Design Code or email 


childrenandtheGDPR@ICO.org.uk 


Privacy statement 

For this call for evidence we will publish responses received from 
organisations but will remove any personal data before publication. We 
will not publish responses from individuals. For more information about 
what we do with personal data please see our privacy notice. 
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Section 1: Your views and evidence 


Please provide us with your views and evidence in the following areas: 


Development needs of children at different ages 


The Act requires the Commissioner to take account of the development 
needs of children at different ages when drafting the Code. 


The Commissioner proposes to use their age ranges set out in the report 
Digital Childhood - addressing childhood development milestones in the 
Digital Environment as a starting point in this respect. This report draws 
upon a number of sources including findings of the United Kingdom 
Council for Child Internet Safety (UKCCIS) Evidence Group in its literature 


review of Children’s online activities risks and safety. 


The proposed age ranges are as follows: 


3-5 
6-9 
10-12 
13-15 
16-17 


Q1. In terms of setting design standards for the processing of children’s 
personal data by providers of ISS (online services), how appropriate you 
consider the above age brackets would be (delete as appropriate): 


Not at all appropriate 
Not really appropriate 
Quite appropriate 
Very appropriate 


Q1A. Please provide any views or evidence on how appropriate you 
consider the above age brackets would be in setting design standards for 
the processing of children’s personal data by providers of ISS (online 
services), 


Our research has not covered this complete age range, but even in our group of 13-17 year olds the 
awareness of personal data and online services, as well as resignation and powerlessness that some 
expressed with regards to data practices varied widely. Whatever age groups and guidelines will be 
generated, they will need to be continually validated in practice. 
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Q2. Please provide any views or evidence you have on children’s 
development needs, in an online context in each or any of the above age 
brackets. 


This lies outside the realm of our research and expertise. 


The United Nations Convention on the Rights of the Child 


The Data Protection Act 2018 requires the Commissioner to take account 
of the UK’s obligations under the UN Convention on the Rights of the Child 
when drafting the Code. 


Q3. Please provide any views or evidence you have on how the 
Convention might apply in the context of setting design standards for the 
processing of children’s personal data by providers of ISS (online 
services) 


We welcome the inclusion of obligations under the UNCRC within the Code, especially in conjunction 
with the UN Guiding Principles on Business and Human Rights, which emphasises "The role of 
business enterprises as specialized organs of society performing specialized functions, required to 
comply with all applicable laws and to respect human rights", since the UNCRC is a direct extension 
of the Universal Declaration of Human Rights acknowledging the particular vulnerabilities of 
children. 


Aspects of design 


The Government has provided the Commissioner with a list of areas which 
it proposes she should take into account when drafting the Code. 


These are as follows: 

e default privacy settings, 

e data minimisation standards, 

e the presentation and language of terms and conditions and privacy 
notices, 

e uses of geolocation technology, 

e automated and semi-automated profiling, 

e transparency of paid-for activity such as product placement and 
marketing, 

e the sharing and resale of data, 

e the strategies used to encourage extended user engagement, 

e user reporting and resolution processes and systems, 

e the ability to understand and activate a child’s right to erasure, 
rectification and restriction, 

e the ability to access advice from independent, specialist advocates 
on all data rights, and 

e any other aspect of design that the commissioner considers 
relevant. 
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Q4. Please provide any views or evidence you think the Commissioner 
should take into account when explaining the meaning and coverage of 
these terms in the code. 


Q5. Please provide any views or evidence you have on the following: 


Q5A. about the opportunities and challenges you think might arise in 
setting design standards for the processing of children’s personal data by 
providers of ISS (online services), in each or any of the above areas. 


Q5B. about how the ICO, working with relevant stakeholders, might use 
the opportunities presented and positively address any challenges you 
have identified. 


Q5C. about what design standards might be appropriate (ie where the bar 
should be set) in each or any of the above areas and for each or any of 
the proposed age brackets. 


Q5D. examples of ISS design you consider to be good practice. 


Q5E. about any additional areas, not included in the list above that you 
think should be the subject of a design standard. 


Responses to Q5A, Q5B, Q5C, Q5D and Q5E are listed below for 
each aspect of design: 


In accordance with the desire by this call for evidence to hear the perspectives and experiences of 
children we have structured our evidence around representative quotes from the participants in our 
youth juries. Juries were audio recorded, with the permission of participants, transcribed by an 
external company, and thematically analysed by us. The selected quotes were chosen based on the 
topics related to this inquiry. 


The outcomes of the ESRC funded CaSMa (Citizen-centric approaches to Social Media analysis 
[http://casma.wp.horizon.ac.uk]) youth juries are summarised in the report “The Internet on Our 
Own Terms: How children and young people deliberated about their digital rights’”. An analysis of an 
initial set of the EPSRC funded UnBias (Emancipating Users Against Algorithmic Biases for a Trusted 


Digital Economy)[https://unbias.wp.horizon.ac.uk/] youth juries was published as contributions to 
the 2017 WebScience conference’ and the 2018 Social Media & Society conference’. 


The participant groups and methodology were as follows: 
° Participants in the CaSMa youth juries were recruited through contacting secondary schools, 
colleges and academies in Nottingham and London and youth centres in Leeds. Recruitment for the 


1 The Internet in Our Own Terms, CasMa project Report, January 2017. 

2 Perez, E., Koene, A., Portillo, V., Dowthwaite, L., & Cano, M. (2017) Young People’s Policy Recommendations 
on Algorithm Fairness. WebSci’ 17 June 25-28 Troy, NY, USA. 

3 Dowthwaite, L., Creswick, H., Portillo, V., Cano, M., Perez Vallejos, E., Koene, A., & Patel, M. (2018). Work-in- 
Progress: UnBias Youth Juries. In Social Media and Society Conference. Copenhagen, Denmark: AC 
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UnBias youth juries was done through Nottingham University links and partners. A total of 108, and 
273 participants (including 13 participants from two pilot youth juries) took part in the CaSMa, and 
UnBias, juries respectively. 

° Methodologically, youth jury participants were asked to consider, debate, and share ideas 
about the future of the Internet while providing useful facts and a safe space to discuss, reflect and 
deliberate about their Internet-related concerns. Topics included the use and awareness of data that 
is collected online, regulation of algorithms, algorithm transparency and children and young people’s 
recommendations for change. The youth juries use scenarios as prompts to encourage discussions. 
The key scenarios relevant for this inquiry are linked to two of the five digital rights for children’: 

- ‘The Right to Make Informed & Conscious Decisions: The Right to Agency’ 

- ‘The Right to Safety and Support’ 


Within the evidence provided below, italic text boxes represent anonymised quotes from the youth 
juries participants. 


> Responses using the arrow shaped bullet points are based directly on the analysis of the 
outcomes of the youth juries 


=" Responses using the square shaped pullet points are additional suggestions from the project 
team. 


Default Privacy Settings 
Key Challenges for children and young people: 


> Some children and young people do not always appreciate the significance and 
importance of data privacy: 


“I'm kind of like not bothered about my privacy because | hate to be a Debbie downer but like 


who’s really gonna be bothered about me? Like, really?” 


> Others had many concerns about their privacy. Many children and young people explained 
how they had taken many steps in using their agency to protect their personal data by 
changing their privacy settings, to ensure that they were as private as possible. 


“Yeah. I’m on, like, every social media where I have always have it put on private, so | have, 
if, if someone wants to follow me, | have to follow them.” 


“Well, | only give my name and my email address, because that’s the only information they 
really need and | don’t want them having my private information because then they can do 
stuff with that, but they can’t really do much with my name.” 


“| only follow my friends and family who are close to me but | sometimes follow celebrities, 
see what’s going on in their lives because | mean there’s quite, they’re quite popular and 
everyone wants to know what’s going on but when it comes to me | hardly share anything 
because | don’t trust people” 


4 Five rights for young people online as formulated by 5Rights https://S5rightsframework.com/the-5-rights/ 


V1.0 20180626 


Opportunities offered by the Code: 


> 


It is encouraging that a lot of children and young people are taking measures to protect their 
personal information when online, however we argue that it should not be up to them to 
have to actively change their settings in order to do this. It was common for the children 
and young people to explain how they felt that it was “weird” and “creepy” for the platforms 
and others to have access to so much of their data. In particular, many had concerns and 
fears about the platforms having access to their location: 


“Yeah, like location data, you don’t want that being sold because that’s sensitive and there’s 
a lot of users that are under the age that Facebook says.” 


“I was going to say places you have been is quite scary because it’s like they have access to 
your proper location and it’s like you said so many companies have access to that, so it’s a bit 
worrying.” 


There were many types of data that some of the children and young people did not wish to 
share such as their email address, phone number, browser history, political opinions and 
information relating to their health. 


Recommended Design Standards: 


> 


In light of the children and young people’s concerns about their location as well as their 
concerns about other types of data such as those listed above, it adds further impetus for 
the need for restrictive default privacy settings to ensure that children and young people are 
aware of what they are signing up to, and which data they are agreeing to share by signing 
up to the app. 


In addition, the concerns around loss of control over personal data, privacy settings that are 
set to minimize data sharing by default are important cybersecurity features for children 
who, due to limited life-experience, have a high potential vulnerability to social engineering 
style cyber-attacks such as various forms of phishing. Spear-phishing targeted at children of 
high-valued targets (e.g. corporate CEOs) could yield access to data that can be used for 
subsequent phishing attacks on their parents. 

For children it is important that strong data privacy settings are the default settings because 
their innate enthusiasm for experimentation means they are likely to sign-up to many apps 
that they might end up using only once, as an experiment. During the sign-up process they 
are unlikely to manage privacy setting before engaging with the new app. If they find that 
the app is not to their liking they are liable to stop using it, but rather than removing all data 
and closing the account they are liable to leave their data on the account at the default 
privacy setting. 


Data minimisation standards 


Key Challenges for children and young people: 


Data minimisation is an important requirement of GDPR (and the resulting Data Protection 
Act 2018) for minimising unintended spreading of personal data, based on the concept that 
the safest way to stop unauthorized access to data is to not hold the data in the first place. 
One of the reasons why ISS might fail to comply with data minimisation is the current 
software engineering practice of code reuse, where code that was developed for one 
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purpose gets uploaded to software libraries and reused for other purposes when a new 
application is developed that requires a similar functionality (e.g. code for implementing a 
sign-up process for an app). In some cases, this development practice might lead to 
unnecessary data collection if features in the original code that are not necessary for the 
repurposed usage are not disabled (e.g. access to phone GPS data by a flashlight app that 
reused code originally written for a camera app with location-based photo tagging). Such 
sloppy code reuse is especially likely to occur in cheap ‘novelty’ apps which are likely to find 
a primary use base in children who are more likely to spend time searching through app 
repositories for ‘joke’ apps. 


Opportunities offered by the Code: 


=" By setting a strong justification requirement for the collection of data the Code offers an 
opportunity to motivate developers to engage in more rigorous design practices which are 
likely to have secondary benefits in cybersecurity. 


Recommended Design Standards: 


=" All data that is collected must be clearly document with purpose justification. 


The Presentation and Language of Terms and Conditions and Privacy Notices 
Key Challenges for children and young people: 


> Terms and Conditions are inaccessible, meaning that many are unaware of what they are 
signing up for when using a particular app. For example, one youth juror pointed out that 
“there’s a lot of words that are really complicated that we kind of don’t know.” 

> Immediate desire to ‘get on’ and use the app seemed to outweigh any consideration of 
Terms and Conditions. For example, one youth juror explained: “/ wouldn’t read it because 
like... it’s just you want to get on the app, so I would just do that.” 

> Many young people expressed how they often felt resigned to signing Terms and Conditions 
due to a lack of alternative: “I just go along with it, | read it and say agree.” One youth juror 
had explained how they had even experimented with not accepting the Terms and 
Conditions, but had found that it did not allow them to access the site: 


“I tried it once on some random website and I scrolled down and | said no I do not agree and 
then it wouldn’t let me click the signup account like signup button until | actually said yes | 
agree.” 


> Some also believed that the companies deliberately made the Terms and Conditions 
confusing so that users are not aware of what they are signing up to: 


“Websites exploit the fact that people don’t look at the terms and conditions so they use it as 
a way to just get information off you.” 


Opportunities offered by the Code: 


> Whilst some youth jurors appeared apathetic and “not bothered” about online Terms and 
Conditions, it is notable that recommendations for how to improve them were made across 
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all juries (see below). This indicates the importance of ensuring that children are consulted 
over such changes as many have valuable things to say. Moreover, feelings of apathy 
expressed by some children and young people may present an excellent opportunity to 
positively address this by taking on board the design recommendations made by those in 
their age group to help to reduce levels of apathy. 


Recommended Design Standards: 


> 


> 


Improve the accessibility of Terms and Conditions by using pictures and videos to engage 
young people. 

Offer different versions of Terms and Conditions, from simple to complex, to allow users to 
choose which version they would like to read. 

Simplify Terms and Conditions- make them more concise, use simple language and a bigger 
font. 

Allow users to ‘personalise’ Terms and Conditions- so that users are able to opt in and out of 
different parts of the agreement. In particular, platforms should be more transparent in 
whether and how their data is being sold. 

Ensure meaningful consent- some youth jurors put forward interesting suggestions, where 
users should have to stay on the Terms and Conditions page for a minimum amount of time, 
to encourage them to read the agreement. 

Use simple bullet points to summarise key points. 

Make them shorter. 


All types of collected data need to be listed in an easy to scan through format (e.g. bullet 
points), including data that does not require consent because it is a legitimate required data 
in order to provide the service (e.g. location data being collected by Augmented Reality 
games like Pokemon-Go where location data is needed in order to correctly interface the 
location of the player with the environment). Non-technical customers, especially young 
game players and their parents, may not be aware of the kind of data that is required to 
enable a service to function. 

Terms and Conditions currently request all permissions in advance, even if some data types 
are not required for the features of the app that the user is actually using. An incremental 
approach would be a rational way to implement one aspect of data minimization. 


Use of Geolocation Technology 


Key Challenges for children and young people: 


> Concerns for safety- that their location may be accessed and made available by some apps. 
> Consent to have the location accessed is conditional to using some apps: One juror 


explained, “yeah, because it’s like things like Snapchat or Twitter, then you can do location or 
you can’t download that app unless you agree to them accessing your location.” 

Concerns can lead to fear and anxiety. As a result, some counteracted any anxieties by using 
their agency to protect data that they saw as private, such as their location, by adjusting 
their privacy settings to ensure that geolocation technology was not enabled. One juror 
also went further by not using an app “...because you can’t make it private”. 


“Yeah, on Snapchat you also have this thing called Snap Maps. So basically, it’s kind of weird 
I think, but you can see where all your friends are by using these little bit emoji’s. But, um, 
you can turn it on and off if you want to. Mine’s off, | don’t... | think it’s a little weird.” 
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“It’s really creepy. So, you can put it on ghost mode so they can’t see where you are. But if, 
say, you had, like, a stalker on social media, they can see where exactly you are...” 


> Others made careful decisions about when and how their location was reported: 


“I mean, like, with twitter you can disable where your locations were when you twitted it so 
I’ve got mine disabled because it’s actually like really really accurate and it could like post the 
name of my tiny village hmm, which would actually be dangerous. And then, like, I’m all right 
with twitting, like, general locations. Like, last week | twitted | was here, because | was, | 
think about two hours after | actually done it. It was like, but that point | was home so, 
nobody is gonna find me” 


Opportunities offered by the code: 


> A potential opportunity is that one youth juror explained that they were aware that “a lot of 
people do it for fun” and “{...] change their location to like America so they've got a ton of 
American adverts and stuff’. If children and young people are some of those that change 
their location, then this is potentially producing false data for the provider. If design 
standards are set for this age group so that geolocation details are not permitted, then this 
may reduce the amount of inaccuracies in the data. 


Recommended Design Standards: 


> Given the concerns around location that were highlighted by the jurors, we suggest that 
children should not have to make these decisions, and that the websites and apps should 
automatically protect the location of children and others by setting their geolocation 
technologies to be disabled. 


“I recommend that the internet should have better security and it shouldn’t let your personal 
data or your location e.g. when you go on Facebook it shows exactly where you are and | 
think those things should be private.” (Written recommendation) 


Automated and Semi-Automated Profiling 
Key Challenges for Children and Young People: 


> Many benefits were recognised by jurors, in offering them a personalised internet service, 
which was recognised as helpful, convenient (“/ like it because it’s quicker!”), and beneficial 
in offering the opportunity to learn new information “And sometimes new experiences can 
be a good thing. Although you might not want information overload, just new things can be 
good as well”. 


> However, the challenges identified included how many had experienced inaccuracies in 
the profiling, where they were shown recommendations that did not interest them, that 
missed the nuances in their searches (for example that they may search on someone 
else’s behalf), and a minority of jurors expressed their concerns that profiling may have 
led them to receive inappropriate recommendations: 
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“Mostly but then sometimes you'll watch the odd video that you wouldn’t really want to 
watch, but then you get loads of recommendations about that certain type that you just 
wouldn’t watch again.” 


“One time | was watching a Peppa Pig movie because my cousin was round and then after 
that it kept coming up with really weird movies about Peppa Pig [...], which was annoying 
because | couldn’t find what | wanted to watch.” 


“But also sometimes I go on like sites about, like, religion because I'm interested in religions. 
And then | get pop-ups that say single Muslim women and things like that.” 


> Another challenge that arose in relation to this theme is that of discrimination. A minority of 
youth jurors were able to identify how profiling might cause harm, in the form of 
discriminatory decisions, towards particular individuals or groups of individuals. For 
example, in one youth jury, a juror pointed out that profiling could harm some vulnerable 
groups, for example migrants: “Well, if you’re, uh, an immigrant coming from a certain 
country they might profile you as someone to look out for”. \n another jury, one juror 
pointed out their concern that their profiling could lead to employers making discriminatory 
decisions about them, based on where they went to school “Because, for example, | went to 
a school that wasn’t really good, and if employers were to know that | went to this school 
then they would probably look at me in a bad way.” 

> During the juries, the children and young people were shown a number of examples of 
algorithmic bias, and many expressed their concerns at how they occurred, and that 
profiling may lead the proliferation of discriminatory views towards certain groups, 
particularly based on race and gender. 

> Some youth jurors felt that the level of personalisation that they were comfortable with 
depended on the context, for example music and entertainment recommendations were 
seen as useful, however the personalisation of newsfeeds was met with concern: 


“If an algorithm is working in a way that if you Googled ‘Trump’ and then it only gave you 
negative or positive articles, then that’s wrong I think.” 


Opportunities offered by the Code: 


= The experiences of the youth jurors above offer an excellent opportunity for the 
development of the age appropriate design of automated and semi-automated profiling, by 
giving platforms the opportunity to address these challenges meaningfully, by allowing the 
user to have more control over how they may be profiled. 

=" A good starting point might be by default no profiling, with the option of choosing the 
degree of personalisation depending on the context (e.g. entertainment versus news). 


Recommended Design Standards: 


> Jurors wanted to have more control over the algorithm, as sometimes the algorithm’s 
profiling of them was inaccurate, or it did not allow them to see other information: 
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“I think you should like be able to... It shouldn’t... instead of letting it assume things, if you 
like want it to be sort of personalised to you, you would be able to give it like basic things like 
how old you are because I think my computer thinks I’m a fifty year old with a Master’s 
degree”. 


The Sharing and Resale of Data 


Key Challenges for Children and Young People: 


> 


Whilst some youth jurors were aware of data sharing and reselling, others were not aware, 
with many believing that the only way that social media companies make a profit is through 
advertising. Others realised that their data was collected, but they did not know that it was 
shared. Some expressed strong disagreement at the selling of their data to third parties, 
particularly as a profit is made from it, as one youth juror pointed out: 


“Yeah, | was just going to say that in some respects, it’s not... | don’t think it’s necessarily 
right for other companies, especially big companies, to make money off my own information. 
Um, yeah, | don’t think that my details and stuff should circulate so that other people can 
make a profit off of it.” 


However, some jurors showed an awareness that companies need to make a profit and 
therefore data selling is a necessary part of it, even if they did not necessarily agree with it: 


“I know it’s like a bit bad but companies like Facebook they wouldn’t exist if they didn’t sell 
your data, so you wouldn’t actually have social media if they didn’t sell your data.” 


Some youth jurors also expressed how they felt disempowered by the platforms’ processes 
of data sharing/reselling, and as a result they did not feel that they had any control over it: 
“L...] but you just brush it off, well can’t really do anything about it.” By setting design 
standards for this age group, it may help the platforms to empower young people and 
improve their sense of agency online. 


Concerns that may arise are that, in similar ways to the design conditions for Terms and 
Conditions, some jurors again expressed apathy towards their data being shared and resold 
especially if they had not experienced any negative consequences directly: 


“Unless it’s affecting my life in a negative way then I don’t think I’m that bothered by it 
really. But if it’s affecting me in a bad way then | would be bothered. But I don’t see how it 
has really so far, touch wood.” 


“I never really knew what it was and I’ve always clicked yes, because nothing’s really 


happened to me if I do click yes.” 


Of those who expressed apathy in relation to data sharing and reselling, some indicated 
their belief that this was because they felt that their data was insignificant: “f...] I’m like one 
person in like millions. | just think I’m a bit irrelevant, | don’t think | matter to them like...” 
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> 


Related to views on apathy, other youth jurors seemed less concerned about the sharing 
and reselling of their data when the data is aggregated, “I think it’s fine, because they're not 
just selling your individual data, there are going to be like thousands of people all mixed to 
an average.” Another juror explained that they turned “...a blind eye” to their data being 
sold on to third parties. When questioned further, they explained that they had done so 
because they were largely apathetic “...you don’t really care that much...” due to their belief 


that the data being shared and sold was not “..super personal”. 


In addition, a few youth jurors expressed that increasing the transparency of platforms by 
knowing where their data was sold, would actually increase their anxiety or concern: 


“It’s like I'd rather not know they'd sell my data because it'd kind of get you scared about 
social media where people are, like, a lot of people wouldn’t like it to be quite a scary thing, 
because they want to use it for, like, good things, for, like, communicating with friends and 


stuff.” 


An additional challenge that was not directly raise by the youth jurors is a lack of clarity 
concerning the actual effect of “privacy settings” on various sites. Especially on social media 
sites, changing privacy setting from public to private frequently only changes the visibility of 
posted content with regards to other platform users, but has no impact on data collection by 
the platform provider. 


Opportunities offered by the Code: 


> 


Setting design standards will heighten transparency so that children are aware of the sale 
and sharing of data when they sign up to apps and websites. It may also reduce anxiety 
amongst children and young people, as some explained how they felt scared and anxious 
that their data was being shared and resold: 


“It’s kind of scary in a way. I think people aren’t aware, so it’s like you could just be sharing 
anything and then they've got access to that.” 


“Well you’re saying it’s collecting all this information, it’s actually quite scary how much, 
even if you put something small online how much they actually can get from it, which makes 
me quite vulnerable in a way from putting something on. And you're thinking oh no one’s 
really going to see it, but actually.” 


Recommended Design Standards: 


> 


Many of the youth jurors expressed anxieties with regard to the types and amounts of 
data shared with third parties, and believed that this should either be stopped, or at the 
very least, that there should be more transparency in relation to what data is shared and 
with whom. 


> Background checks on third party companies. A couple of youth jurors suggested 
carrying out background checks. One explained: “when it comes to data selling, do 
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background checks on the companies wanting to buy. Think less about the money 
making and more about the safety of people.” (Written recommendation) 

> Increase awareness of Data Selling/Reselling: “Take into consideration whose data is 
being shared. Children are less aware and they don’t understand.” 

> Reduce Data Selling/Reselling: “Limit the amount of data they can collect” 

> Platforms to give more choices: “Yeah, we should have a say in what goes where.” 


> In addition, a couple of jurors suggested that the user should be emailed to update them on 


what data had been shared and with whom, with the option of the user taking their own 
actions to contact the company to remove their data if they wished: 


“You could like have the email system and then it will email you and say, we have sold your 
details to this company, to prevent this from thingy, contact this company via this and just 
make it so it’s a long process, so people are less likely to do it so they still make money but 
still do it as an option, so people can still opt out, but it’s just a bit more difficult.” 


To avoid confusion regarding privacy setting on platforms the Code should define clear 
terminology that distinguished between settings that regulate “public visibility of content” 
and settings that regulate “privacy of content”, which includes limits on data access by the 
platform provider. 


Strategies Used to Encourage Extended User Engagement 


Key Challenges for Children and Young People: 


Many youth jurors acknowledged their excessive use of the Internet, while they believed others 
“tend to fail to realise that”, but they still find it very difficult to self-regulate when using digital 
technologies. 


Extended use creates over dependence and impacts on children’s wellbeing and emotional state: 


> 


‘Addiction’ to social media and online games 


“{...] it’s really addictive and you say you're not, but you really are because you just can’t stop 


ace 


“Um, when the advertisement comes up it does that and you've got a good score so you 
wanna keep going and then it keeps happening, but you keep going and you get addicted.” 


“[...] they do things where you have to wait so and so hours for a new life, or [...] for the 
action to be completed. So you’re constantly checking the game to see if your action has 
been completed or not, and that’s what makes it so addictive”. 


Sleep deprivation 


“I'll be trying to get to sleep and then I'll get a message and you tell yourself, no, just ignore 
it. And then five minutes later, lIl be like, oh, no, I’ve gotta see what it says, and then [...] you 
find yourself having a conversation...” 
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“Well, like on a night before bed, you might go on YouTube and watch some videos, and then 
you'll have suggestions up by the side, you'll be like, [...] I'll go to sleep after this video. Then 
you're like, oh, but what about this one? And that’s just like a continuous cycle, before you 
realise it’s two o-clock in the morning, you have to be up in five hours.” 


> Annoyance from unhealthy dependence on social media 


“I’m on Twitter [...] | was trying to go to sleep [...] all of a sudden, my phone started notifying 
me about these people [...] following me and every single time I just had to like wake up in 
the night [...] check and then it would come again, and again, again, again. And then | was 
almost [...] | was really about to smash my phone or something because it was annoying. So, | 
just decided to turn off my phone.” 


> Social media platforms potentiate peer-group pressure 


Children and young people felt under lots of pressure by their peer group to reply on the 
spot to the messages they get on social media, adding that it is what it is expected from 
them: “There’s no option but reply to the message.” When asking young people why they 
cannot ignore the messages, answers included: 


“People might think that you trying to deliberately ignore them [...].” 
“{...] if you don’t reply to something it seems offensive [...].” 


“| think it’s as well as the pressure to reply it’s kind of like the expectation that as a teenager 


you’re...to be up late and you’re going to be online...if you're up late you're cool [...]." 


>  Self-isolation and impact on children and young people’s social skills 


“You shut yourself in and then you just become more isolated. Erm, ‘cause you don’t get out 
much, you don’t like see people, you don’t socialise and then that sort of affects you in future 
[...] you get a job, you get to be out and about socialising with people, and you’re not gonna 
have any skills to do that.” 


> Impact on self-esteem 


“It makes people doubt their style of life, and their way of life, because somebody else’s is, in 
their opinion, better.” 


“It just massively breeds insecurity | mean I’m the eldest of three girls so sisters, and when 
like it’s, it’s you’re always kind of insecure around this time anyway [...] And what it’s kind of 
done for my sisters l'Il say they’re getting into that awkward age it’s this massive comparison 
to people who aren’t really people. [...], they don't see it and then they have this ridiculous 


expectation of themselves [...].” 


> Feeling excluded and depressed 


“| feel like, sometimes, when people haven’t been invited to like, a party and all their friends 
have and they see it all over Facebook, [...] and they have to look at everyone’s pictures of 
them having a good time, [...Jand they weren’t wanted there [...] that can be quite hard for 


some people.” 
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“Sometimes looking at peoples’ newsfeed or your newsfeed is slightly depressing [...] | would 
have wanted to come to that’ and then they seem all really happy [...] get me depressed 
looking at them”. 


Opportunities offered by the Code: 


The inability of children, teenagers in particular, to self-regulate exposure to digital 
technologies, impulsivity, peer-pressure, low self-esteem and the impact on their wellbeing 
and mental health, prevent them from using technology that is in their best interests*° 


The recent recognition by the World Health Organisation (WHO) of “gaming disorder” as a 
disease (ICD-11)’ has now made the NHS in the UK offer treatment to children suffering 
from that medical condition’. We see this as a timely opportunity for the Commissioner to 
work in partnership with the Department of Health and Department for Digital, Culture, 
Media and Sport, as well as the Department for Education to tackle and prevent health 
issues related to ‘persuasive design‘, making ‘no-persuasive by design’, a priority in their 
agendas. 


Besides, jurors considered the overdependence to some digital platforms, games or 
applications as an issue the UK government should take more responsibility in order to 
protect online users’ health: 


“[...] when you use a website, | think maybe at the top it should tell you the reasonable 
amount of time you should spend on that website for your health. So, the government could 
do something like that to help people. [...] like the time they're using, and how you're 
impacting them, [...]., you don’t know anything, you're just going to keep on using, but if they 
tell you a little bit about it, it could help.” 


Also, participants believed the industry to have a commercial interest in users’ 
overdependence behaviour and that a third party should intervene to make industry 
compliant addressing the ‘addictive’ elements of their platforms: 


“It’s all about the sheer greediness of these companies, they should, but they won’t, [...] I 
think if something were to be done about it, some sort of third party would have to step in 
[...], and say this isn’t right, you really shouldn’t be doing this. Or perhaps even force them to 
stop, or something like that [...]” 


“These companies have way too much power over how addictive their games can be. I’ve 
seen Flappy Bird is one example [...] And they could try to do it, try to change it by adding 
regulations of, you’re only allowed half an hour on your device at a time [...]” 


Related to policy-making, a juror compared how laws to prevent addiction to digital 
technologies could be developed and applied the way gambling laws have: 


> Pp 18-23, Digital Childhood, B Kidron and A Rudkin, December 2017 
5 See Chapter 4 (pp 27-31), Disrupted Childhood, B Kidron, A Evans and J Afia, 5Rights, June 2018 


7 https: 


icd.who.int/dev11/l-m/en#/http%3a%2f%2fid.who.int%2ficd%2fentity%2f1448597234 


8 Gaming addiction can be treated on the NHS after it is declared a medical disorder, C. Hymas, the Telegraph, 
15 June 2018 
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“It works in the same way as like the new gambling laws do where you set a limit before you 


start playing about how much money you're going to spend.” 


Recommended Design Standards: 


> Young people were very eager to provide recommendations to find solutions ‘by design’, to 
the over-dependence they felt to social media and gaming apps. The most popular 
suggestions included time and posting limits on social media, more control to users to opt 
on/off online status, removing the elements that leads to addiction to services, etc.: 


> Time limited usage: 


“[...] you should have a time cap on your games, so if you set like a two-hour time cap within 


a 24-hour period [...]. So then you don’t have to be like too addictive.” 


Time restrictions were also met with scepticism as one juror pointed out that a regulation 
would not be adhered to: “f...] No one really listens to authority that much, especially when it 
comes to things like this [...].”It was also noted that this kind of regulation might discriminate 
against those with a learning disability whose time on the internet may aid their 
communication skills: 


“[...] there’s young people with learning disabilities such as autism who actually need the 
internet to communicate with other people [...] that’s a bit unfair to that person [...].” 


> Time notifications: 


A juror comparing online over-dependence to gambling addiction, suggested adding time 
notifications to help users manage the time they spend online. 


“[...] maybe a notification which popped up saying, you’ve been on this app for however long. 
Or like a timer at the bottom which says how long you've been on, because | think that does 
create a greater awareness [...].” 


> Limited number of posts/messages: 


“So, having posting limits, so be it on Facebook, [...] especially Snapchat [...], if you're 
exceeding the limit, you shouldn’t be able to do any more.” 


> More control to the users to show their on/off online status on social media: 


“To have a choice to show whether you are online and whether you have read a message to 
remove that pressure to reply.” 


> Removal of persuasive design features on devices and services used by children: 


“They should, like, cut it down, like the addiction part of it, so like you can get not addicted 
[...]” 
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User Reporting and Resolution Processes and Systems 
Key Challenges for Children and Young People: 


> A minority of the youth jurors described cases in which they reported offensive online 
content to the platform provider, to varying degrees of success. One youth juror went 
through the process of reporting online content to a social media platform, and had this 
successfully dealt with, following clear communication between the platform and the user 
“the person would private message you on Instagram, personal privately message you and 
then you will explain what happened [...]”._ Despite this youth juror’s positive experience of 
online reporting systems, they also believed that the platforms “underestimate” the impact 
of keeping accounts up, despite requests to take them down: 


“| think that sometimes they underestimate it a little bit because it can be an account where 
it’s like maybe an account you made years ago where you put picture on it that are a little bit 


embarrassing so then Instagram will just think that it’s not that big of a deal. But then to you 
obviously it’s something different than the way they see it.” 


> Distrust in the effectiveness of platforms’ resolution processes: 
A minority of youth jurors described not only their frustration at seeing repeated 
advertisements for things that they were not interested in or that they had seen on 
numerous occasions before, but also that they did not feel that reporting changed this, as 
they continued to see the adverts, “/t went away for a bit and then it came back.” 


Opportunities offered by the Code: 


> Another youth juror explained that they had seen inappropriate content online but did not 
know how to report it, suggesting that introducing age appropriate design to user reporting 
and resolution processes may be a beneficial way to empower young people, and help them 
to report online content that they do not wish to see. 

> The findings above from the youth juries present a number of opportunities to improve user 
reporting and resolution processes, which will provide an excellent opportunity to empower 
young people, and for companies to forge a meaningful relationship with children and young 
people where they feel listened to. 


Recommended Design Standards: 


Children and Young people recommendations: 
> One youth juror pointed out the importance of the user being able to flag up online content 
that they did not agree with on all platforms and websites: 


“You could have like, can you not have like online securities? And you have things where you 
click on something and then you can speak to somebody like a report put in and stuff to make 
it more, because on some websites they don't even have it, if I’m honest with you, so if you 
have more of those available where people can actually see it and it will be anonymous to an 


extent.” 
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There needs to be concise, engaging and aged-appropriate instructions for children to report 
online related issues. Formats might include: short videos or diagrams (e.g., including 
cartoons or comics). If using bullet points, maximum five and using plain language. 


The Ability to Understand and Activate a Child’s Right to Erasure, Rectification and 
Restriction 


After presenting a scenario about The Right to Remove®?°, depicting children and young people 
feeling embarrassed and/or uneasy about pictures and/or content they had posted online, but then 
wanting to delete them, jurors expressed: 


> 


The will to be protected from leaving “permanent digital foot-prints from their ‘immature 


selves’ttas well as to safeguard their wellbeing. 


“I know we’re talking about things you do when you're younger but some people might just 
put a picture up that’s not ... nothing wrong with it, but they might get people saying stuff 
about it anyway, because they might be bullied or whatever. So I think they should have the 
right to take it down if they're getting upset [...], it’s like their life’s gonna be ... made worse 
by that picture still being there [...]” 


Key Challenges for Children and Young People: 


> 


> 


Lack of knowledge on how to erase content and when seeking support, social media 
platforms not addressing children’s requests of deletion: 


“[ .] And then when I contacted em, they said they couldn’t do nothing about it and they just 
keep the pictures up... they don’t tell ya how to delete your Instagram, only how to make an 
Instagram” 


In a minor discussion relating to this theme, one juror explained that they’d “/...] never 
thought about it” while another felt that it would not be particularly relevant to them: 


“No, because I don’t think there’s anything on there that matters that much. There’s pictures 
from when I was 13 or whatever that are a bit awkward but there’s nothing on there that is 
that important.” 


One juror pointed out their frustration that they were not able to fully deactivate their 
Facebook account: 


“I find it how bizarre when you try to deactivate your account on Facebook you can’t 
actually delete it completely because when | tried to deactivate mine it said, OK, you’ve 
deactivated it but you can always reactivate and I didn’t reactivate, but I still get emails from 
Facebook saying, oh yeah.” 


Disbelief that content posted online would be effectively erased: 


° https://5rightsframework.com/the-5-rights/the-right-to-remove.html 


Remove.mp4 
11 Pp 28 The Internet in Our Own Terms, CasMa project Report, January 2017. 
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“Even if they get removed from the internet, someone is bound to have it, so you're not [...] 
you're never sure if it’s completely gone”. 


Some of the jurors agreed on the difficulty to delete their online content completely. In 
particular, concerns about screenshots being “impossible” to be fully deleted from 
cyberspace were shared: 


“it’s hard to get it deleted because [...] if you get a picture removed on Facebook someone 
could have screenshotted it, quickly screened it and it’s on their computer so it’s like you 
could get something removed but really once something’s out there it might not ever be 
gone completely... | don’t think that’s ever possible...” 


Some jurors shared real situations that evidenced “dangers” of screenshotting and also 
expressed their concerns about the lack of awareness people have on this matter: 


“I think that issues that people don’t know the dangers of posting something. [ .] | have a 


friend who had a very like private conversation with someone she knows and they actually 
screen shot the conversation and she mentioned like a lot of embarrassing private things on 
there. And once he posted it on his own Facebook she was kind of outcast from school and 
bullied to the extent that she had to move to another school. So | think people need to be 
well advised on how dangerous it actually is to post something.” 7” 


Opportunity offered by the Code: 


Setting design standards offers an excellent opportunity to empower children and young 
people by helping them to learn about their right to erasure. 


Recommended Design Standards: 


> 


Platforms should address children’s request to erase content “straight away” once reported: 


“I think they need to ... there needs to be a quicker reaction, as soon as something’s 
reported to like to actually get it taken down like straight away. Rather than oh | reported 
this image three days ago and it’s still there”. 


To focus on educating children about the possible issues of posting content online: 


“[...] | think the main issue is teaching people what not to post before it’s too late, and 
before they post it rather than focusing on deleting what’s already been done”. 


Few children are posting for archival purposes; a default of time limited posting would be 
the appropriate design guideline. 


Q6. If you would be interested in contributing to future solutions focussed 
work in developing the content of the code please provide the following 
information. The Commissioner is particularly interested in hearing from 


12 Pp 30 The Internet in Our Own Terms, CasMa project Report, January 2017. 
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bodies representing the views of children or parents, child development 
experts and trade associations representing providers of online services 
likely to be accessed by children, in this respect. 


Name: = G behalf of Horizon Digital Economy Research, University of Nottingham) 


Email: Es 1 ottingham.ac.uk 


Brief summary of what you think you could offer 

Horizon is a Research Institute at The University of Nottingham and a Research Hub within the RCUK 
Digital Economy programme. Horizon brings together researchers from a broad range of disciplines 
to investigate the opportunities and challenges arising from the increased use of digital technology 
in our everyday lives. on the ESRC 
funded CaSMa_ project (Citizen-centric approaches to Social Media analysis 
[http://casma.wp.horizon.ac.uk]) to promote ways for individuals to control their data and online 
privacy and the EPSRC funded UnBias (Emancipating Users Against Algorithmic Biases for a Trusted 
Digital Economy)[https://unbias.wp.horizon.ac.uk/] project for raising user awareness and agency 
when using algorithmic services. led the research of the CaSMa and UnBias 
projects. erformed research for the UnBias 
project. An important part of this work included the facilitation of ‘youth juries’ — workshops (similar 
to focus groups) with 13-17 year old youths designed to identify experiences, concerns and 
recommendations about online services. The protocol to run youth juries is available as an Open 
Educational Resource. 


In addition to our work directly with young people on their experiences of engaging with ISS, the 
UnBias team is also involved with preparing a Science and Technology Options Assessment report on 
governance frameworks for algorithmic accountability and transparency for the European 
Parliament, and participates in the European Al Alliance. 


Further views and evidence 


Q7. Please provide any other views or evidence you have that you 
consider to be relevant to this call for evidence. 
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Section 2: About you 


Are you: 


A body representing the views or interests of children? 
Please specify: 


A body representing the views or interests of parents? 
Please specify: 


A child development expert? 
Please specify: 


A provider of ISS likely to be accessed by children? 
Please specify: 


A trade association representing ISS providers? 
Please specify: 


An ICO employee? 


Other? 
Please specify: 

Horizon Digital Economy Research institute, University 
of Nottingham (EPSRC funded UnBias project) 


Thank you for responding to this call for evidence. 
We value your input. 
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